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ABSTRACT 

We contrasted portrayals of the achievement gap on 
the North Carolina state mathematics assessment 
for students with disabilities (SWD) using two 
different ways of identifying this group of students. 
The first method used students' disability status in 
third grade as the basis for identifying SWD, and 
then tracked the achievement gap between SWD 
and students without disabilities (SWoD) across 
grades. With our second method, we allowed 
student disability status to change at each grade, 
so that only students who were receiving special 
education services in a particular year were 
considered members of the SWD subgroup for that 
year. 


Achievement Gaps 

One of the overarching goals in many 
educational reform efforts including the No Child 
Left Behind Act of 2001 (NCLB, 2002) and the 
recent Common Core State Standards Initiative 
(www.corestandards.org) has been high levels of 
achievement for aN students. The examination of 
achievement gaps between reference and focal (at 
risk) groups of students has been widely used as a way 
to identify groups for whom the educational system 
is falling short of this goal, as well as a way to then 
monitor progress in "closing the gap." 

Although the term "achievement gap" 
appears often in the educational lexicon (Chudowsky, 
Chudowsky & Kober, 2009; Konstantopoulous & 
Hedges, 2008; Lee, 2010) there are multiple ways of 
describing these gaps. NCLB requires states, districts, 
and schools to disaggregate and report the percent 
of students reaching grade level proficiency in 
reading and mathematics by gender, race/ethnicity, 
economic status, and English proficiency level. Here, 
achievement gaps are defined as differences between 


reference and focal groups in percentage of students 
in each group reaching or exceeding grade-level 
proficiency. For example, the Center on Educational 
Policy has reported that differences of 30 or 40 
percentage points are common between the percent 
of SWD scoring at or above grade level proficiency 
compared to SWoD on states' large scale assessments 
of mathematics and reading achievement (Chudowsky 
et al., 2009). 

A second way of examining achievement gaps 
is simply looking at differences in means between 
groups. As an example, a recent report (National 
Center for Education Statistics, 2013) concerning 
achievement score trends on the National Assessment 
of Education Progress (NAEP) found that the average 
mathematics score for 9-year-old Black students grade 
had increased substantially since 1971. Although 
White students had also made gains across time, they 
were not as large as the gains for Black students. The 
report concluded that the White/Black achievement 
gap in mathematics on the NAEP for this grade had 
narrowed because the difference between the means 
for the two groups was smaller than it had been in 
1971. 

A third way of characterizing achievement 
gaps is with effect sizes (Bloom, Hill, Black, & Lipsey, 
2008). Effect sizes standardize the characterization 
of group mean differences by expressing them in 
ratio to the standard deviation of observed test 
scores. For example, a mean difference of 15 points 
between two groups on a test would be viewed quite 
differently if the standard deviation of scores on the 
test for the two groups was 25 versus 100 points. 
When characterizing achievement gaps, effect sizes 
permit gaps obtained on different measures to be 
more appropriately compared, because the gaps are 
all reported using the same metric. To illustrate, an 
achievement gap effect size of -0.5 can be interpreted 


Page 1 




as indicating that the mean for the focal 
group (e.g., economically disadvantaged 
students) fell one half standard deviation 
below the mean for the reference 
group (e.g., students not economically 
disadvantaged). 

Each of these ways of 
characterizing the achievement gap 
produces a different portrayal of 
group performance differences. The 
three approaches appear frequently in 
public reporting and publications that 
characterize achievement gaps or trends 
in achievement gaps across time or 
grades, although in most cases, effect 
sizes are the preferred metric (Bloom et 
al., 2008). 



Figure 1. Size of achievement gap between students with and without disabilities, 
in terms of percent of students reaching grade level proficiency, for longitudinal 
and cross sectional samples. 


The Achievement Gap for Students with Disabilities 

Low academic achievement for students 
with disabilities (SWD) has been a longstanding 
concern (Blackorby, Chorost, Garza, & Guzman, 2005; 
McDonnell, McLaughlin, & Morison, 1997). In recent 
years, addressing the achievement gap between SWD 
and SWoD has been one of the most problematic 
aspects of NCLB. For example, Eckes and Swando 
(2009), in a study of three states, found that the most 
frequent reason for schools failing to make Adequate 
Yearly Progress was that too many students in the 
SWD subgroup had scored below grade level so that 
this subgroup did not meet the percent proficient 
benchmarks set by their state. 


Although there are likely many reasons 
schools encounter difficulties closing the achievement 
gap for this subgroup, Ysseldyke and Bielinski (2002) 
raised concerns that identifying the SWD subgroup 
on the basis of student's annual participation in 
special education masks achievement progress by this 
subgroup. Disability classification is less stable than 
many of other student characteristics that are tracked 
in terms of achievement gaps, such as student gender. 
Each year students who are academically successful 
are more likely to exit from special education, and 
students who are experiencing academic difficulty 
in general education are more likely to enter special 
education. Using large-scale assessment data 
from one state, Ysseldyke and Bielinski reported 
special education turnover rates of 
approximately 20% per year. They found 
that achievement gaps between SWD 
and SWoD were smaller when a stable 
subgroup of SWD, defined on the basis 
of special education membership at one 
point in time, was tracked rather than 
reconstituting membership in the SWD 
subgroup each year. 

In our study, we examined how 
achievement gaps differed (whether 
stated as differences in percent 
proficient, mean score differences, or 
effect sizes) when a stable, longitudinal 
sample of students comprised the 
SWD subgroup versus a sample that 
changed annually depending on special 
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•Longitudinal sample 

•Cross sectional 
sample by grade 


Figure 2. Size of achievement gap between students with and without disabilities, 
in terms of difference in mean scores, for longitudinal and cross sectional samples. 
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•Longitudinal sample 

•Cross sectional 
sample by grade 


Figure 3. Size of achievement gap between students with and without disabilities, 
in terms of effect size, for longitudinal and cross sectional samples. 


education participation for that year. The total sample 
was 92,045 students, all of whom were in the third 
grade for the first time in 2001, were not retained 
across grades three to seven, and had participated in 
the general assessment in mathematics at least once 
during the five-year time period of the study. In third 
grade, approximately 12% of the total sample was 
identified as a student with a disability. These students 
comprised the longitudinal SWD sample. We then 
contrasted the achievement gap for this stable sample 
of SWD with cross sectional samples at each grade, 
where membership in the SWD subgroup was defined 
by participation in special education that year. 

Figure 1 depicts the size of the achievement 
gap in terms of the difference in the percent of SWD 
reaching grade level proficiency for a grade compared 
to SWoD. In grade three, the longitudinal and cross 
sectional SWD samples were the same students 
(because each group was defined as those students 
who were in special education in grade three). About 
81% of SWoD had mathematics scores that placed 
them at or above grade level proficiency; only 56% 
of the SWD subgroup achieved scores in this range. 
Thus, there was an achievement gap of 25% in terms 
of the students in each group scoring in the proficient 
range or higher. Across grades, the achievement gap 
fluctuated up and down somewhat, and the values 
for the two different ways of constituting the SWD 
subgroup tracked these year-to-year fluctuations 
closely. However, in every year after third grade, the 
gap was more extreme when the SWD subgroup was 
cross sectional. 

Figures 2 and 3 depict SWD achievement gaps 


at each grade based on the difference 
between mean scores for SWD and 
SWoD. In Figure 2, each gap is the 
difference between the mean score for 
the SWoD and the SWD subgroups for 
that grade and way of defining the SWD 
subgroup. As in Figure 1, after grade 
three, the mathematics achievement 
gap was more extreme (i.e., larger) 
when the SWD subgroup was based 
on a cross sectional sample. In Figure 
3, the differences between the means 
for the SWD and SWoD groups have 
been converted to effect sizes at each 
grade by dividing the mean difference 
between the SWD and SWoD groups 
by the pooled standard deviation of 
mathematics scores for that grade. 

More extreme gaps result in effect sizes that are 
more negative. Again, the gaps were generally 
larger when the SWD subgroup was based on a cross 
sectional rather than longitudinal sample. Note 
also that, although the achievement gap grew wider 
with increasing grade level in Figure 2 for both SWD 
subgroups, when the gap was stated as an effect 
size, it narrowed from grade six to seven. This 
discrepancy in depictions of the achievement gap 
as widening or narrowing depending on whether 
mean differences or effect sizes are used is a result 
of the fact that effect sizes adjust mean differences 
for changes in the natural variation in scores across 
grades. The natural variation in scores increased in 
later grades, and interpreting achievement gaps in 
light of this change in the dispersal of student scores 
provides a more accurate context for interpreting the 
achievement gap. 

In sum, regardless of how the SWD subgroup 
is comprised, there is a sizable gap between the 
achievement of students with and without disabilities 
across grades. However, because entrance and 
exits from special education are related to student 
achievement, failing to take this factor into account 
when examining achievement gaps may lead to 
mistaken conclusions about schools' effectiveness 
in addressing SWD achievement needs. It is also 
important to recognize the achievement gap can be 
portrayed in multiple ways, such as differences in the 
percent of students reaching proficiency and effect 
sizes, and results from the different approaches may 
yield discrepant results. 
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